Skip to content

docs(changelog): Flaky Tests Test Collections#146

Closed
samgutentag wants to merge 2 commits into
mainfrom
sam-gutentag/changelog-flaky-tests-test-collections
Closed

docs(changelog): Flaky Tests Test Collections#146
samgutentag wants to merge 2 commits into
mainfrom
sam-gutentag/changelog-flaky-tests-test-collections

Conversation

@samgutentag
Copy link
Copy Markdown
Member

@samgutentag samgutentag commented May 29, 2026

What shipped

Test Collections — named groups of tests within a repository, each with their own settings, quarantine policies, and scoped views. Upload test results to a collection directly from the Trunk CLI using the collection's short ID, and see tests, uploads, and configuration scoped to that collection across the Flaky Tests app.

Source

  • Eng PRs: trunk-io/trunk2#3583 (CRUD core), #3598 (UI polish), #3660 (collection-level quarantining), #3678 (QA fixes)
  • Linear: TRUNK-18209 (no absorbed duplicates; sub-tickets TRUNK-18012, TRUNK-18048, TRUNK-18066, TRUNK-18067 are the eng work)
  • Date basis: latest source PR mergedAt — #3678, 2026-04-22

Files wired (4 sites)

  1. changelog/2026-04-22-flaky-tests-test-collections.mdx (new entry)
  2. docs.json (Changelog tab, 2026 group)
  3. changelog/index.mdx (April 2026 section)
  4. flaky-tests/changelog.mdx (new April 2026 section)

Docs link target: https://docs.trunk.io/flaky-tests/get-started/test-collections

🤖 Generated with Claude Code

@samgutentag @claude
docs(changelog): Flaky Tests Test Collections
45e8cb5 
Add changelog entry for Test Collections — named groups of tests within
a repository, each with their own settings, quarantine policies, and
scoped views. Upload results to a collection via the Trunk CLI using the
collection's short ID.

Source eng PRs: trunk-io/trunk2#3583 (CRUD core), #3598 (UI polish),
#3660 (collection-level quarantining), #3678 (QA fixes).
Linear: TRUNK-18209.
Date basis: latest source PR mergedAt (#3678, 2026-04-22).

Wired into: changelog/2026-04-22-flaky-tests-test-collections.mdx,
docs.json, changelog/index.mdx, flaky-tests/changelog.mdx.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@mintlify
Copy link
Copy Markdown
Contributor

mintlify Bot commented May 29, 2026 •
edited
Loading

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
trunk 🟢 Ready View Preview May 29, 2026, 5:27 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@samgutentag
Copy link
Copy Markdown
Member Author

samgutentag commented May 29, 2026

Verification status (2026-05-28): live

Verified: customers can use this. Ready to publish.

  • Eng PRs: trunk-io/trunk2#3583 (CRUD core, stacked merge of children #3545-#3550), #3598 (UI polish), #3660 (collection-level quarantining), #3678 (QA fixes). All merged and intact on main.
  • Flag: none. No frontend gating flag (useFeatureFlag/useFlag) was found across the stacked children #3545-#3550 or the follow-up PRs. The only LaunchDarkly reference is a backend dual-write/webhook flag in #3660's test plan, with no named identifier and no preserved legacy code path.
  • Signals: stacked-merge recursion into child PRs (no flags), e2e flags.json (no collection flag), code search for useFeatureFlag+collection (0 hits), Slack (design discussion only, no flag-state chatter). Linear records production deploy tags v154 through v160.

Ready to publish.

@samgutentag samgutentag added changelog PR touches the changelog (auto-generated drafts, hosting, formatting, indexing). ready to merge Verify docs PR: customers can use this. Ready to publish. labels May 29, 2026
@samgutentag @claude
docs(changelog): correct CLI upload flag to --test-collection-short-id
a32b92a 
verify-docs-against-code found the upload flag is --test-collection-short-id
(analytics-cli/cli/src/upload_command.rs), not --collection.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@samgutentag
Copy link
Copy Markdown
Member Author

samgutentag commented May 29, 2026

Code verification (2026-05-28): 2 confirmed / 1 contradicted (corrected in this PR) / 0 ambiguous / 0 unverifiable

Claim Verdict Source
Upload results to a collection via its short ID confirmed upload_command.rs:74-81
CLI upload flag is --collection contradicted (fixed) upload_command.rs:74-81
Admins manage collections; members can view confirmed test-collection/index.ts:141-315

The original entry said the upload flag was --collection. Source says the flag is --test-collection-short-id (clap field test_collection_short_id, env TRUNK_TEST_COLLECTION_SHORT_ID). Corrected in commit a32b92a. Note: the underlying docs page flaky-tests/get-started/test-collections.mdx still shows trunk flakytests upload --collection <short-id> and should be fixed separately before this changelog publishes.

Source #1 — Upload via short ID / CLI flag name (confirmed + contradicted)

File: trunk-io/analytics-cli/cli/src/upload_command.rs#L74-L81

    #[arg(
        long,
        env = constants::TRUNK_TEST_COLLECTION_SHORT_ID_ENV,
        help = "Optional test collection short ID to attach to the uploaded bundle for collection-aware ingestion.",
        required = false,
        num_args = 1
    )]
    pub test_collection_short_id: Option<String>,

Reasoning: The uploader accepts a test collection short ID, which confirms the "upload via short ID" claim. The flag uses clap's long attribute with no explicit rename, so the field test_collection_short_id becomes --test-collection-short-id (kebab-case). There is no --collection flag. The collection-short-id strings found in trunk2 are data-testid UI selectors, not the CLI flag.

Source #2 — Admin manage / member view RBAC (confirmed)

File: trunk-io/trunk2/ts/apps/frontend/src/lib/services/flaky-tests/test-collection/index.ts#L141-L315

// list / get
const { orgId } = await requireSessionOrgReadAccess({ session });
// create / update / delete
const { orgId } = await requireSessionOrgWriteAccess({ session });

Reasoning: Read operations (list collections, get collection, get details) are guarded by requireSessionOrgReadAccess and mutations (create, update, delete) by requireSessionOrgWriteAccess. This confirms the documented split: any org member with read access can view collections; only members with write access (org admins) can create, edit, or delete them.

@samgutentag samgutentag added the needs eng review verify-docs-against-code: at least one claim contradicts source. label May 29, 2026
@samgutentag
Copy link
Copy Markdown
Member Author

samgutentag commented May 29, 2026

Closing in favor of #137, which already publishes this same Test Collections changelog entry AND fixes the docs page (removes hidden: true, corrects the upload flag to --test-collection-short-id). This PR only added the changelog entry, so #137 supersedes it. Verified during the changelog backlog sweep.

@samgutentag samgutentag deleted the sam-gutentag/changelog-flaky-tests-test-collections branch May 29, 2026 05:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

changelog PR touches the changelog (auto-generated drafts, hosting, formatting, indexing). needs eng review verify-docs-against-code: at least one claim contradicts source. ready to merge Verify docs PR: customers can use this. Ready to publish.

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@samgutentag